Adaptive optimal training of animal behavior

Abstract Neuroscience experiments often require training animals to perform tasks designed to elicit various sensory, cognitive, and motor behaviors. Training typically involves a series of gradual adjustments of stimulus conditions and rewards in order to bring about learning. However, training protocols are usually hand-designed, relying on a combination of intuition, guesswork, and trial-and-error, and often require weeks or months to achieve a desired level of task performance. Here we combine ideas from reinforcement learning and adaptive optimal experimental design to formulate methods for adaptive optimal training of animal behavior. Our work addresses two intriguing problems at once: first, it seeks to infer the learning rules underlying an animal's behavioral changes during training; second, it seeks to exploit these rules to select stimuli that will maximize the rate of learning toward a desired objective. We develop and test these methods using data collected from rats during training on a two-interval sensory discrimination task. We show that we can accurately infer the parameters of a policy-gradient-based learning algorithm that describes how the animal's internal model of the task evolves over the course of training. We then formulate a theory for optimal training, which involves selecting sequences of stimuli that will drive the animal's internal policy toward a desired location in the parameter space. Simulations show that our method can in theory provide a substantial speedup over standard training methods. We feel these results will hold considerable theoretical and practical implications both for researchers in reinforcement learning and for experimentalists seeking to train animals

Simultaneous Multi-Vessel Subacute Stent Thromboses in Zotarolimus-Eluting Stents

Despite its low incidence, stent thrombosis (ST) is one of the most dreaded complications of percutaneous coronary intervention. Endeavor (Medtronics Europe SA) is a new zotarolimus-eluting stent (ZES) with a favorable safety profile that was reported in early and ongoing trials. However, few lethal stent thromboses related to this new drug eluting stent (DES) have been reported. We experienced a case of simultaneous subacute ZES thromboses, 6 days after stent implantations in the proximal left anterior descending artery and the proximal right coronary artery (RCA)

Influence of Lamina Terminalis Fenestration on the Occurrence of the Shunt-Dependent Hydrocephalus in Anterior Communicating Artery Aneurysmal Subarachnoid Hemorrhage

Recently, it was reported that fenestration of the lamina terminalis (LT) may reduce the incidence of shunt-dependent hydrocephalus in aneurysmal subarachnoid hemorrhage (SAH). The authors investigated the efficacy of the LT opening on the incidence of shunt-dependent hydrocephalus in the ruptured anterior communicating artery (ACoA) aneurysms. The data of 71-ruptured ACoA aneurysm patients who underwent aneurysmal clipping in acute stage were reviewed retrospectively. Group I (n=36) included the patients with microsurgical fenestration of LT during surgery, Group II (n=35) consisted of patients in whom fenestration of LT was not feasible. The rate of shunt-dependent hydrocephalus was compared between two groups by logistic regression to control for confounding factors. Ventriculo-peritoneal shunts were performed after aneurysmal obliteration in 18 patients (25.4%). The conversion rates from acute hydrocephalus on admission to chronic hydrocephalus in each group were 29.6% (Group I) and 58.8% (Group II), respectively. However, there was no significant correlation between the microsurgical fenestration and the rate of occurrence of shunt-dependent hydrocephalus (p>0.05). Surgeons should carefully decide the concomitant use of LT fenestration during surgery for the ruptured ACoA aneurysms because of the microsurgical fenestration of LT can play a negative role in reducing the incidence of chronic hydrocephalus

Measurement of the azimuthal anisotropy of Y(1S) and Y(2S) mesons in PbPb collisions at root s(NN)=5.02 TeV

The second-order Fourier coefficients (v(2)) characterizing the azimuthal distributions of Y(1S) and Y(2S) mesons produced in PbPb collisions at root s(NN) = 5.02 TeV are studied. The Y mesons are reconstructed in their dimuon decay channel, as measured by the CMS detector. The collected data set corresponds to an integrated luminosity of 1.7 nb(-1). The scalar product method is used to extract the v2 coefficients of the azimuthal distributions. Results are reported for the rapidity range vertical bar y vertical bar < 2.4, in the transverse momentum interval 0 < pT < 50 GeV/c, and in three centrality ranges of 10-30%, 30-50% and 50-90%. In contrast to the J/psi mesons, the measured v(2) values for the Y mesons are found to be consistent with zero. (C) 2021 The Author(s). Published by Elsevier B.V.Peer reviewe

25th annual computational neuroscience meeting: CNS-2016

The same neuron may play different functional roles in the neural circuits to which it belongs. For example, neurons in the Tritonia pedal ganglia may participate in variable phases of the swim motor rhythms [1]. While such neuronal functional variability is likely to play a major role the delivery of the functionality of neural systems, it is difficult to study it in most nervous systems. We work on the pyloric rhythm network of the crustacean stomatogastric ganglion (STG) [2]. Typically network models of the STG treat neurons of the same functional type as a single model neuron (e.g. PD neurons), assuming the same conductance parameters for these neurons and implying their synchronous firing [3, 4]. However, simultaneous recording of PD neurons shows differences between the timings of spikes of these neurons. This may indicate functional variability of these neurons. Here we modelled separately the two PD neurons of the STG in a multi-neuron model of the pyloric network. Our neuron models comply with known correlations between conductance parameters of ionic currents. Our results reproduce the experimental finding of increasing spike time distance between spikes originating from the two model PD neurons during their synchronised burst phase. The PD neuron with the larger calcium conductance generates its spikes before the other PD neuron. Larger potassium conductance values in the follower neuron imply longer delays between spikes, see Fig. 17.Neuromodulators change the conductance parameters of neurons and maintain the ratios of these parameters [5]. Our results show that such changes may shift the individual contribution of two PD neurons to the PD-phase of the pyloric rhythm altering their functionality within this rhythm. Our work paves the way towards an accessible experimental and computational framework for the analysis of the mechanisms and impact of functional variability of neurons within the neural circuits to which they belong

Shaping the Information Channel: Molecules, Cells, and Experiments

An important and ubiquitous question in the physics of biological systems is how an information transmission channel is shaped and optimized by a careful exploitation of the structural details of the underlying system. This dissertation explores three instances of this central question, at the levels of molecules, cells, and scientific experiments. The first chapter is focused on the molecular gateway of cellular signaling, where a ligand concentration reflects some extracellular condition, and a receptor works as an information channel by binding the ligand and consequently activating the downstream signaling pathway. In this sense, the ligand-receptor binding event is the initial information channel of the signaling process, setting an upper bound on the final amount of information transmitted. We investigate how the information flow through the ligand-receptor binding can be optimized by the choice of the kinetic parameters, and how it is limited by various constraints of the cellular environment. Once the signal is initiated, it needs to be relayed until it reaches the final target in the cell. Such signal transduction pathways are built on a network of specific protein-protein interactions, and one of the important determinants of interaction specificity is shape complementarity. In the second chapter, we aim to characterize the statistical properties of the ensemble of proteins in the cell, in terms of the shapes of protein surfaces. We study the intrinsic dimensionality of the space of surfaces, and discuss how it is linked to the properties of individual protein surfaces, revealing the non-trivial organization of the shape space. The third chapter concerns the optimization of the design of a scientific experiment, viewing the experiment as an information channel through which the scientist collects data about the natural world. Specifically, we consider a behavioral neuroscience experiment where the aim is to infer the psychometric function, which governs the stimulus-dependent decision-making behavior of an animal. We demonstrate how the experimental design can be optimized to reach the desired precision of measurement with a minimal amount of data, using an adaptive, closed-loop algorithm that selects the most informative stimulus at each trial

The regulatory reform system and policy coordination in Korea

노트 : Government Publications Registration Number 11-1051000-000566-0